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Making UNIX investments Baye 


In the 80's the open systems debate raged; open vs 
proprietary, de-facto vs de-jure standards, UNIX System 
V vs other *IXes, pragmatism vs principle, binary vs 
source portability, free vs fee, connectivity vs 
interoperability etc. 


The life of IT decision makers was difficult. Should | buy 
open or proprietary, should | buy UNIX and if so which 
one, what technologies and standards should | invest in? 
If the goal was to invest in a UNIX solution, there were 
many real and imagined risks. The key issue being which 
hardware and software technology should | invest in? 


Standardisation, at Last? 


Recent work in standards by nauetny > bodies, = apactiently 
X/Open with SPEC 1170, offers the long discussed 
single UNIX standard. This standard delivers the open 
systems goals of portability of applications, 
interoperability of systems and transportability of skills. If 
all UNIX systems are now the same, surely this makes an 
IT decision maker's life much easier when selecting a 
UNIX solution. 


The risk areas associated with a UNIX decision in the 
80's have shifted in the 90's. An IT decision maker no 
longer has to worry about UNIX and Open Systems being 
the wrong fundamental decision. There is substantial 
evidence that UNIX solutions do deliver. In the 90's we 
focus on how to make the investment in UNIX give the 
maximum return. 


The question faced when making a UNIX investment is 
"How do | maximise the value of the investment in UNIX 
technology and minimise the associated costs and 
risks?". \f all applications are portable because of 
standardisation, and interoperability is demonstrable, 
and skills broadly transportable, then what do | use to 
select among the UNIX contenders? 


| Table 1: IDC Midrange Cost Study 
Area Europe US 
Hardware costs 15.8% 8.8% | 
Hardware Maintenance 7.0% 44% 
Systems software 5.5%|_ 3.3% 
Systems software support 3.3% 1.5%| 
Tools and applications 12.6% 16.4% 
Staffing- Systems Management 46.8% 65.7% 
Total 100.0% 100.0% 
NOTE: Source IDC UNIX Midrange Cost Study 

Average over 4 US and 6 European vendor's systems 


This paper eenenee that the most “Sianitieant 
contributor to the cost_of ownership, and therefore the 
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return _on_ investment for UNIX systems, is in fact the 
systems management capability of a UNIX solution. 


This paper looks at work by IDC and other consultants 
that shows the staffing cost associated with systems 
management is the largest variable cost in a modern 
UNIX solution. 


Carefully evaluating the systems management capability 
of prospective UNIX systems is the key to making UNIX 
investments pay-off, and is in fact more significant than a 
focus on systems performance or price performance. 


SPEC 1170 and the Standardisation ws 
UNIX 


The SPEC 1170 standard was announced in September 
1993 and endorsed by 75 vendors at that time. SPEC 
1170 specifies a single UNIX operating system by 
specifying a standard set of Application Programming 
Interfaces or APIs (originally 1,170 in number). 


Operating systems that comply with all SPEC 1170 
standards will be eligible to carry the UNIX trademark. 
This standard was published by the X/Open company on 
October 14th 1994. Test suites to verify SPEC 1170 
compliance by an operating system are due for 
completion by year end 1994. We will see SPEC 1170 
compliant UNIX systems in 1995. 


This means that at last all UNIX systems will be 
demonstrably the same, at least from an applications 
viewpoint. SPEC 1170 is the latest, and largest, of the 
recent attempts to standardise the applications interfaces 
in UNIX. Because of its widespread support, SPEC 1170 
is now the pre-eminent standard superseding AT&T's 
SVID, IEEE's POSIX, X/Open's XPG3 and XPG4, and 
OSF's AES. 


SPEC 1170, and the cluster of standards that it 
envelopes, makes purchasing a UNIX system that much 
easier. We can buy only systems that will comply with 
SPEC 1170. This makes SPEC 1170 mandatory but not 
a differentiator. 

What Doesn't SPEC 1170 Offer? | 


cite cease ReeeRCR UNOS ORCHESTRAS, 


The aim of SPEC 1170 is applications portability, it does 
not address all areas of operating system function. SPEC 
1170 is attempting to guarantee applications portability 
by guaranteeing interfaces. So specifying SPEC 1170 
will buy portability of applications at a source code level. 


This is analogous to specifying that UNIX shail be a 
sphere of 10cm diameter. All UNIX implementations will 
have the same external appearance and shape (SPEC 
1170 or a 10cm sphere so to speak), but if | cut the UNIX 
(sphere) in half | am not guaranteed that the internal 
construction and materials employed will be identical. 


Does this matter to UNIX systems buyers? Certainly 
preserving the investment in applications is a major 
benefit. Having UNIX implementations substantially 
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similar is an advantage. But there are key areas where 


UNIX systems differ because vendors want to 
differentiate their products and users want to address 
some of the weakness of generic UNIX. The end result is 
that UNIX systems are the same, because they comply to 
the same standards, but also different because of 
commercial and user demands. The differences tend to 
manifest themselves in the following areas: 


> The underlying processor and systems architectures. 


> Leading edge functionality such as objects and client 
server where standardisation is less solid. 


» Systems administration and management capability . 


» |naseeming contradiction to SPEC 1170, applications 
support. 


All these issues are important in a UNIX decision. Which, 
though, have the most impact on maximising the return 
on a UNIX investment? 


What Elements Impact the Costs Associated with 
UND? 


Pee RONEN ARR HONRIRRONNONOE 


Recent work done by consulting firms in the area of cost 
of ownership of UNIX and other similar systems is 
painting an interesting picture, and helps answer the 
question of what areas are most significant in a UNIX 
investment. 


Table 2: Selected Computer Staffing Cost Findings 
Source Finding on Staffing Costs 


Gartner Group 83% of the 5 year cost of PCs is 
labour 


LAN Admin and help desk makes up | 
67% of corporate LAN Costs 


51% of enterprise computing costs 


Forrester Research 


IDC 1993, "Mainframe and 
LAN Costs-of-Use" [are staffing. 
Gartner Group - "AGuide | Labour costs account for 71-75% of 


For Estimating Client / Server | total costs. 
Costs" 


John Chisolm Group, 
"Client-Server Computing: 
Conquering the Enterprise” 


ITG 1994, "Cost of 


Labour accounts for 85% of client 
server computing costs in 1993. 


— 


60.2% of the cost of PC/LANs and 


Computing" 31.7% of the cost of Mainframes is 
personnel. 

Xephon 1992, "The Dinosaur | Personnel is 60% of the 5-year cost 

Myth" of UNIX systems. 


Table 1 summarises an IDC study. This work shows that 
people costs are the most significant element in UNIX 
system cost of ownership, ranging from 47 to 66 percent 
of costs. 
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This is supported by data from a number of other 
sources, as we can see in Table 2. The data in Table 2 
supports the IDC findings in regards to cost of 
ownership. We will return to this topic, but first let us 
explore the nature of systems management. 


Systems management is a form of insurance. Without 
due diligence, a system will fail, or be unable to perform 
its tasks. The investment made in a systems operation 
and management is made either explicitly or implicitly. By 
this | mean that systems management costs are either 
planned or occur in an ad-hoc manner. 


So where does all the money spent on systems 
management go, and why. A survey taken by UniForum 
is outlined in Table 3. 


Table 3: A Profile of Systems Management Activity 


Area of Systems Management Activity Time (%) 
Directly helping people Ty 36 % 
Systems maintenance * Ty 24 % 
Personal professional development 11% 
Installation and configuration * 9% 
Ensuring and implementing the future * 7% 
[Management 5% 
Software development * 5% 
Backup and restore * 3% 


[NOTE: * areas impacted by systems management technology 


Source: UniForum Monthly, August 1994, Page 38 - Article 
"Conceming Your Career", Author - Jim Johnson 


NOTE: Results of a time/expenditure survey conducted at the 1992 
USENIX Large Installation Systems Administrator's (LISA) 
conference. Compiled by Rob Kolstad 


— 


Table 3 shows that a large portion of systems 
administration time clearly goes to doing what should be 
done, helping users to exploit the investment in systems, 
software and infrastructure. 


Over half a systems manager time, 53%, is directly 
related to tasks that one could characterise as pure 
systems management. These tasks can be impacted by 
systems management technology. 


We will refer to systems management as the planning, 
performance and management of activity in the following 
areas: 


> Network Management: Managing problems, 
performance or operational issues which arise in the 
network. 

> Performance / Capacity Management: Monitoring 
resources for overload, assisting in determining future 
capacity needs . 
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> Configuration Management: Managing the addition, 
deletion, modification, distribution and inventory of 
systems resources. 


> Change Management: Planning, distributing, tracking 
and installing changes to systems, applications, and 
data. 


» Problem Management: Detecting, reporting, tracking 
and resolving problems. 


> Data Management: Managing the backup, restore, 
archive, and distribution of data. 


> Print Management: Managing print queues and 
devices. 


> Workload Management: Managing the scheduling, 
priority, and distribution of work across single or 
multiple systems. 


> Resource Accounting: Managing the usage, 
accounting and billing of resources. 


> Security Management: Managing authorised access 
to resources . 


Is There a Difference in UNIX Systems Management 
Capability? 


Today there is there a difference between UNIX Sustenis 


in the area of systems management capability. There are | 


many initiatives under way to attempt to standardise 
UNIX systems management regimes and 
single system manager to manage multiple systems from 
different vendors. 

DME, COSE, and POSIX Systems Management 
Committees are examples in the standards area. IBM's 
NetView/6000, CA's UniCenter and Tivoli's Wizdom are 
examples of products that have a similar aim. 


Standardisation is progressing, but today there are very 
real differences in the manageability of UNIX systems. In 
this environment we have to make a immediate decision 
on essentially proprietary technology with an eye to the 
emerging standards of the future. 


Hardware 
HW Maint. 
Op Sys 

OS Maint. 
Tools Apps 


Staffing 


LK 2K 3K 4K 


Figure 1: ue UNIX Midrange Cost Structure 
($ per user per year) 
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to allow a | 


Table 4: IDC Midrange Cost Study Cost Variance | 
($ per user per year - Western Europe) 
Area |_Best Worst Vary %o 
Hardware costs 320 480 160 11% 
Hardware Maintenance 130 230 100 7% | 
Systems software 70 250 180 12% 
eae ee 
Systems SW support ) 210 210 14% 
Tools and applications 560 560 0 0% 
Staffing | 870) ‘1700 830| 56% 
Total (Best and Worst) 1950| 3430 1480 100% 
ea pam os 
| Actual Vendor Totals 2190 3100 910 68% 
NOTE: Source IDC UNIX Midrange Cost Study 


There is no data available to prove the link between 
staffing costs and systems management activity and the 
role systems management technology can play in 
reducing costs of staffing. 


| ask the reader to draw from their own experiences with 
UNIX systems management to substantiate the link 
between cost savings and the systems management 
capability of UNIX and add-on systems management 
software. 

Cost Leverage 


cannnsnnecntes 


We have, to date, looked at the summary aa data from 
the "IDC Midrange Cost Study”. Let us look in some more 
detail at this data. In particular we will examine the data 
on a per vendor basis. Figures 1 and 2 contain the 
vendor cost data. 


Dm Ge Ch) (Sb sy 


1K 2K 3K 4K 


Figure 2: Western Europe: UNIX Midrange Cost 
Structure ($ per user per year) 


Le 


The IDC Study focused upon the actual cost of owning 
midrange UNIX systems. The study surveyed 230 user 
sites in Europe and the US. | have removed all vendor 
names. Note that staffing costs are defined as "..the 
costs of having an IT department and refer only to the 
share of staffing costs actually related to running the 
UNIX midrange systems." 
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The data in Figures 1 and 2 clearly demonstrate that 
while systems may look good when comparing direct 
purchase costs, the costs of ownership is substantially 
impacted by the costs of people which is the cost of 
systems management. Table 4 summarises the 
variances seen in the cost data. 


Tables 4 and 5 consider the best and worst individual 
cost elements for the vendors analysed by IDC. The 
differences between best and worst costs can be 
considered to be an indication of the maximum cost 
reduction possible in a particular area. The % column 
indicates the percentage contribution that a particular 
cost area contributes to the difference between best and 
worst cases or possible cost reduction. 


Table 5: IDC Midrange Cost Study Cost Variance 
($ per user per year - US) 

Area |_ Best | Worst Vary % 
iHardware costs 180 240) 60} 3% 
Hardware Maintenance 70 160 90 4% 
Systems software 20) ~ 430) | 101 «5% 
Systems SW support 0 100} 100 4% 
Tools and applications 400 400| 0 0% 
Staffing 1050 3000 1950 3% 
Total (Best and Worst) 1720] 4030 2310} 100% 
Actual Vendor Totals 1880 3780| 1900 82% 
NOTE: Source IDC UNIX Midrange Cost Study 


Tables 4 and 5 show there is substantially more 
variability in the cost of staffing (systems management) 
than in any other cost element. We can draw two clear 
conclusions from this work: 


> The cost of staffing, (systems management) is the 
largest element in the costs of UNIX Systems 
ownership 


> The cost of systems administration is the area that 
offers the greatest potential for cost savings when 
considering purchase of a UNIX Systems. 


Impact of Key Differentiators on UNIX Investments 


Previously we discussed four key areas where systems 
UNIX systems can be differentiated, how do these relate 
to the analysis of costs we have just gone through? 


The Underlying Processor and Systems Architectures 


The IDC study indicates that there is little variability in 
systems prices. The lack of major difference in hardware 
pricing due to the competitive nature of the market. This 
is to be expected as this is the outcome that UNIX users 
have long desired. 


The choice of processor will impact the purchaser more 
in the future, when the next decision on hardware is 
made. It is at that time that an option to change vendor, 
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or technology can be considered. At system replacement 
time, a prudent initial choice will allow a purchaser a 
range of hardware choices because of the success of 
that technology in the market. 


The data in Tables 4 and 5 indicate that at best 11% of 
possible cost savings are available via prudent hardware 
selection. 


The hardware technology selection is a critical decision 
in a UNIX purchase. The long term viability of the 
hardware technology is particularly critical as it impacts 
future costs, future hardware choice, and availability of 
solutions. But, the area of hardware costs has 
traditionally received too much attention. 


eens iT decision neal are Sencidne to exploit feuding 
edge technologies to reduce costs in development., 
operation, and maintenance. The impact on a UNIX 
purchase can be considered to be the same as for 
applications which are discussed below. 


Systems Administration and M 


As we have seen, systems management capability can 
provide over 50% of the potential cost savings available 
when making a UNIX decision. 


ort. 


ARNOT ES 


Without applications, middleware, application enablers, 
and software tools, UNIX solutions would be of no use to 
purchasers. The availability of applications that meet 
your business needs is a key issue. 


SPEC 1170 will decrease the need for a certain solution 
to dictate a hardware or operating system technology. 
Today's commercial reality is, though, that the most 
prominent platforms attract the most applications. 


Selecting an application solution is the most critical 
decision in any IT purchase, but for the purposes of this 
paper we are assuming that the right solution has been 
found prior to optimisation of the U UNIX platform. 


LAA ERR AER 


Conclusions and Recommendations 


Based upon our discussion above we can conclude: 


> Systems Management costs account for over half the 
costs of ownership and over half the potential for 
savings in modem UNIX platforms. 


This being the case, it is recommended that: 


> UNIX purchasers should invest approximately half 
their evaluation effort assessing the manageability of 
the various UNIX solutions on offer. 


For Copies of the IDC Cost-of-use papers contact: 


| David Noble, IDC Australia Phone +61-2-922-5300 


ee 


Plug’N’Play Unix 


Andrew McRae 


MITS Real Time Ltd. 
2/37 Waterloo Rd 
North Ryde 
andrew @ mega.com.au 


This paper presents the design and implementation of a Plug and Play system for 
FreeBSD, a freely available version of BSD Unix7, based around PCMCIA cards (PC- 
Card). A brief introduction of the PC-Card architecture is given, followed by a descrip- 
tion of the FreeBSD implementation. It describes the internal workings of card recogni- 
tion, and the automatic installation and configuration of the cards into the Unix kernel, 
and how this affects system configuration and setup. 


1. Introduction. 


The evolution of Laptop PCs has been rapid and impressive. New hardware features such as power 
management, low power LCD screen displays and low voltage high performance CPUs and memory have 
increased the power and functionality of mobile computers significantly. One such development has been 
the widespread acceptance of the Personal Computer Memory Card International Association (PCMCIA) 
standard, a standard for connecting small circuit cards to computing equipment, now renamed the PC-Card 
standard (which is a whole lot easier to say than P-C-M-C-L-A). 


Unfortunately, Laptops almost exclusively use DOS/Windows, which has notoriously poor support 
for networking and communications. Virtually all PC-Card products are centered around the DOS/Windows 
world, and emerging standards such as Exchangeable Card Architecture (ExCA) and Execute In Place 
(XIP) are almost totally geared towards a DOS/Windows environment. 


Unix, on the other hand, has generally been a non-migratory beast, usually residing on machines that 
are too heavy to lift let alone carry on a plane. With the advent of the BSD ports of Unix to the PC environ- 
ment, it is now possible to take advantage of the Laptop PC technology to have a mobile Unix environ- 
ment. FreeBSD is one such freely available port. It seemed logical to integrate the PC-Card technology into 
FreeBSD as a way of leveraging Unix into the mobile computing world, and the result is presented in this 


paper. 


2. PC-Card Overview. 


The PC-Card architecture was originally designed as a memory card system, allowing easy transfer 
of semiconductor based mass storage. The standard was revised (Release 2.01) in 1991 to incorporate 
peripheral devices such as communication cards, network cards etc. The next major revision (Release 2.1) 
standardised support for such devices such as modems. Support has grown for the system such that it is 
now seen as a standard I/O bus for desktop as well as mobile computers. 


In its basest form, the PC-Card architecture can be thought of as simply another peripheral bus con- 
nection standard, but it has been designed with a number of features that make it interesting: 


: A very small form factor so that cards may have the physical dimensions of a thick credit card. 
Whilst all PC-Cards have the same width and height, the depth of the card may vary from 3.3 mm 
(Type I) to 10.5 mm (Type III). Some cards may have an extended length to allow extra room for 


+ Unix is a trademark of someone at the moment, but [ forget just who it is this week. 


peripheral devices etc. 

° Environmently and physically the card is robust, being completely cased in a mechanically strin- 
gently specified surrounding. 

° The card and connector is designed for manual insertion and removal, even while the socket is pow- 
ered up (‘hot-swapping’). The connector is keyed to prevent reverse installation, and is specified for 
up to 10,000 insertion and removal cycles. 


° Every card contains a Card Information Structure (CIS), describing in a Card Metaformat the exact 
type, capabilities and interfacing requirements of this particular card. Thus an inserted card may be 
queried to determine what kind of card it is and how it should be interfaced. The CIS consists of a 
series of self-describing configuration tuples. 


The PC-Card architecture is a natural contender for implementing a true Plug and Play system, since 
every card can be easily inserted or removed, and every card can be uniquely identified and interfaced as 
required. 


2.1. PC-Card Interfacing. 
Figure | shows a typical Laptop implementation of a PC-Card interface, where a slot/socket con- 
troller is used to interface to two separate card slots. 


ines PC Card 


Enable 


jl 


Figure 1: Slot Interface 


System bus 


Slot 


Controller 


The slot controller is typically a device such as an Intel 82365 or Databook TCIC. The controller 
(along with the support circuitry) provides monitoring of the slot connectors, generation of card events such 
as card insertion or removal, power control, bus buffering enabling, address decoding and interrupt steer- 
ing. The system bus connects to the cards through the intermediatary of the slot controller. 


2.2. Memory Architecture. 

PC-Cards, once connected to the system, are accessed similar to other devices via the memory and 
V/O bus. Each PC-Card has the capability of two distinct memory address spaces, Common and Attribute. 
The slot controller has a number of memory windows, each of which translates system addresses to corre- 
sponding address areas in one of the two address spaces on the PC-Card (as shown in figure 2). 


Each PC-Card slot has 26 address lines, allowing a total addressing capability of 64 Mbytes for both 
Common and Attribute memory. Attribute memory locations only exist on the even address bytes, and is 
intended to contain the card’s CIS and other configuration registers. Every card must respond to Attribute 
memory accesses (though a neat trick in the CIS tuples effectively allows a card to contain no separate 
Attribute memory). Attribute memory can therefore only be accessed via byte wide memory references (the 
odd byte is ignored on 16 bit accesses). 

Common memory is accessed using standard 16 bit or 8 bit memory references, and is intended to 
contain the shared memory areas of the card that the system can access. 
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Figure 2: Memory Mapping 


If the card is an I/O card (as opposed to a memory-only card), then the system I/O bus may be used 
to access up to 64K I/O ports on the card. Cards (within their CIS) indicate the number of I/O lines that are 
decoded within the I/O address space; cards may decode I/O port addressing themselves, or allow the slot 
controller to decode the I/O addresses and enable the slot accordingly. 


Interrupts may be generated by cards, and slot controllers have interrupt steering capability, allowing 
the system to configure each card’s interrupt to appear on different interrupt lines. 


The combination of memory and I/O address windows and interrupt steering allow (given a compli- 
ant card configuration) to relocate and manage the resources required for each card dynamically, without 
address pre-allocation or presetting. This is very different from the ISA model, where the configuration of 
the memory, I/O and interrupt resources is at best a complicated and restrictive exercise. 


2.3. PC-Card Metaformat. 


The organisation of the card’s CIS is the key to the plug and play nature of the PC-Card architecture. 
The structure is a series of self-describing tuples (see figure 3), each of which contains a code and a link to 
the next tuple. The link is effectively the length of the data following, allowing easy traversal of the whole 


CIS structure. 
ae 
[| 


Figure 3: Metaformat Tuples 


The PC-Card standard has defined a number of tuple codes which are used to identify the card envi- 
ronment such as power requirements, memory blocks available, I/O addressing etc. Each tuple definition 
has an optional variable sized data block associated with it that contains parameters for the particular con- 
figuration item being described. Not all cards need to contain all the tuple codes; a minimum subset is 
defined for all cards, however. 


The following table provides a sample set of tuple codes: 


dis 
Code Use ] 
00 Null tuple (ignored) 
Ol Describes device parameters for common memory devices 
10 Checksum 
ll Link to tuple list in Attribute memory 
Nea) Version data describing manufacturer etc. 


1A Configuration data header 

1B Configuration entry 

21 Function identification 

22 Functional extension (extra data for e.g modems) 
FF End of tuple list 


Cards may contain multiple configuration entries, each of which defines different card setup parame- 
ters such as I/O address decoding, memory arrangement etc. The system software will select one of the 
configuration entries and then enable that entry by writing a specified value to a configuration control regis- 
ter in attribute space. The location of this register is given in one of the CIS tuples. 


The decoding and understanding of the CIS is one of the more complex areas of PC-Card processing, 
especially when card vendors produce incomplete or inaccurate CIS entries (not an uncommon problem). 


3. FreeBSD Integration. 


The challenge of integrating PC-Card technology into FreeBSD is not so much of ‘Can it be done?’, 
but more ‘How to get the most out of the technology?’. The one existing driver that incorporated some PC- 
Card support (a Network driver) was written for one specific slot controller, and worked on only one kind 
of Network card, and everything was done within the network driver. No insertion or removal was handled, 
and there was no method of allocating dynamic resources such as memory and I/O port allocation etc. 


The general plan was to develop an implementation with the following goals: 


: Provide a user-level configuration file containing enough information to allow card recognition and 
setup without writing any card-specific code. 

: Support the ‘hot’ insertion and removal of cards, enabling or disabling the card drivers as needed. 

. Dynamically allocate system resources such as memory areas, I/O ports and interrupts from a pool of 
available resources. 

. Support multiple types of slot controllers. Different Laptops have different slot controllers, and the 


idea is to provide a consistent user or kernel level interface to all controllers. 


* As much as possible, use existing drivers. This is a realistic goal, since most cards emulated ISA 
devices quite closely. 


° Have some fun with my new Laptop. 


Extended goals for the PC-Card implementation was to allow some research into plug and play con- 
cepts, and support mobile communications via modem, packet radio, wireless LAN etc. 


3.1. Overall Design. 


Figure 4 shows an overall view of the PC-Card interface to FreeBSD. The general idea was to pro- 
vide a number of interacting entities that would be extendable and allow new ideas to be incorporated with- 
out having to re-work the whole system. The two key elements are a user level daemon (‘pemciad’) which 
reads a configuration file (‘/etc/pemcia.conf ) describing the cards and associated configuration data, and a 
socket/slot device driver which interfaces the socket controller to the system. The driver and daemon com- 
municate via a typical read/write/ioctl interface. 

Part of the configuration action of the daemon is to exec external programs to configure installed 
cards, such as running ifconfig etc. The daemon is normally inactive, and waits on a select to the slot driver. 
When a card is installed or removed, the select comes true and the daemon queries the driver for the partic- 
ular event. Each slot is assigned a separate minor device number, which allows each access to each slot 
using standard tools such as cat, dd and hexdump. 


Config 
File 


Support 
Programs 


PCMCIAD 


Slot driver 


Figure 4: Overall Design 
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3.2. Slot Driver. 


As discussed, the slot driver is accessed via a set of ioctl calls and read/write calls. The ioctl calls are 
used to manage the slot resources such as memory and I/O windows, and provide hooks for allocation of 
interrupts and drivers to the slot. By default, a portion of ISA memory address space is allocated and shared 
between all slots for use in a read/write call. When the read or write entry for the driver is called, a spare 
memory window for that slot is configured with the ISA memory address as the host address, with the file 
offset providing the card memory space address. The read/write call is then performed as a simple memory 
copy. If the size of the request is too large for the memory window, the request is broken in smaller blocks 
and the memory window is reconfigured for each block. 


The use of read/write for the card memory interface means that the slot driver does not need to know 
anything about the card’s CIS structure, nor about the reading or writing of any of the card’s configuration 
registers existing in the Attribute memory space. User programs can access these registers simply by doing 
an lseek followed by a write. 


The slot driver also maintains a state for each slot, which includes the contexts for the I/O and mem- 
ory windows, the interrupt number assigned to the slot etc. 


3.3. Card Driver. 


Once a card is installed, eventually a driver is assigned to the card. The driver is a normal device 
driver, such as for a serial UART port or network controller. A BSD Unix driver normally is only set up at 
system configuration time, at which time each driver’s probe routine is called to test whether a device exists 
at the desired address. If a probe is successful, then the driver’s attach routine is called to complete the 
device setup. PC-Card drivers are similar, except a probe at system startup will fail since the card configu- 
ration as not been set up by the daemon, or a card may be plugged in after the system is started. 


Once a driver is probed, it is assumed (naturally) to always exist at the probed location and interrupt 
number. No normal BSD Unix driver expects to have its device untimely ripped from its socket’s connec- 
tor! This of course can happen using PC-Card technology, but at this stage little work has been done on the 
existing drivers to incorporate this change. Current drivers rely on a device reset to ensure the hardware is 
set up. Conversely, when a card is removed, the best that can happen at the moment is that the daemon will 
execute a support program disabling the device, such as executing an ifconfig xx0 down in the event of a 
network card removal. 


The latest release of FreeBSD allows kernel loadable modules; this would mean easy device driver 
loading and unloading. Unfortunately, this is not allowed with some of the more interesting drivers such as 
network drivers. 


3.4. Kernel Changes. 

Little change has been made to the core FreeBSD kernel; the ISA routines have been extended to 
allow lookup of a kernel device table using a name and unit number, and some early changes allowed 
device probes to be called after system startup. 


Some miscellaneous changes were made to make it easier to install generic devices, such as allowing 
the physical Ethernet address to be assigned via an ioctl to the network driver. 


3.5. Daemon and Configuration File. 


Most of the hard work is done by the pemciad program, which parses the configuration file and 
attempts to install the right driver for that card, which has been hopefully configured at the expected mem- 
ory and I/O address with the correct interrupt number. This involves reading and decoding the CIS tuples, 
and matching the card name/version to one in the configuration file, selecting a valid configuration from the 
CIS tuples, and trying to match it with the free resources available, and then enabling the card resources, 
and then assigning a driver to the card. 


A typical configuration file would look thus: 


# 

# Sample configuration file. 
# 

# Pool parameters. 

S 

lo 0Ox2F8 - 0x360 

lng S 6 8 9 10 LS 

memory 0xd4000 96k 


# 

# Card database. 

# 

card "RPTI LTD." "EP400" net # NE2000 clone 


ether 0x110 

config 0x30 "ed0Q" 5 

contig 0x31 “ed1l" 6 

insert ifconfig $device physical Sether 


card “RIPICAA" "RC144ACL" tty 
config 0x21 *sz0l” 10 


device net 
insert ifconfig $device bean 
remove ifconfig $device down 


device net 
insert ifconfig $device bean-2 
remove ifconfig $device down 


device tty 
insert echo start getty here 


remove echo stop getty 


A pool of free card resources is defined, followed by a card database and different device descrip- 
tions for each instance of the different device classes. Once a card is recognised and installed, the daemon 
remembers the card setup even if the card is removed, since once the driver is probed and attached, the 
driver’s resources are allocated permanently and so those resources are not available to other devices. Re- 
inserting the card will then re-allocate the same resources as before. 


A typical series of events following a card insertion is: 


1. The slot driver receives a card event interrupt, signalling a card insertion. 

2. The card is reset and powered up, and the card event flag for the slot device select is set true. 

ey The daemon (if it is waiting on the select) will register a change (via select) and read the new card 
status. 


4. The card’s CIS is read and decoded. 


The CIS version and manufacturer name is matched against the card database from the configuration 
file. If none is found, no more processing is done. 


6. The configuration entries from the CIS are matched against the entries in the file, and one selected 
that matches the free resources. 


ds. The memory, I/O and interrupt resources are allocated to the card, and the memory and I/O contexts 
programmed for that slot. 


8. The slot driver is called to probe the selected driver and assign the driver. If the probe succeeds, then 
the support commands for that particular card and device class are executed. 


Card removal is a little simpler, as the configuration entries do not need to be searched or matched. 


4. Results. 


The design as described has been implemented on FreeBSD 1.1.5.1 and FreeBSD 2.0 on a range of 
Laptops. Whilst the overall objective has been successful, in some cases it has highlighted what needs to be 
developed to get the most out of the PC-Card technology (see later). Network cards and serial cards are the 
predominant use of the system so far, but conceivably any PC-Card can be utilised to a greater or lesser 
extent. The work to date has been submitted to the FreeBSD core team, and should appear in FreeBSD 2.1. 


One impressive demonstration is to start pinging another host, then remove the network card (causing 
ping to return ‘no route to host’ errors) and insert the card again, allowing ping to continue as if nothing 
had happened. 


Some problems have arisen, some of which are solvable by more development on FreeBSD, and 
some which are related either to limitations within the PC-Card technology or standards, or vendor’s imple- 
mentations. 


In particular, the following conclusions were reached about the PC-Card technology: 


: Incorrect or incomplete CIS implementations are a major pain in the neck. Under DOS/Windows 
these are often fixed by the vendor supplying a proprietary ENABLER program designed to set up the 
card so that it can be successfully used. It is unlikely that the vendor would supply an equivalent 
FreeBSD program. 

< Whilst the memory and I/O address mapping is independant of the ISA bus, the interrupt line uses 
the PC interrupt controller, and so the device configuration must take this into account when allocat- 
ing/probing devices. 

: The PC-Card standard is not. Specifically, the CIS Metaformat does not provide tuple configuration 
standards for various I/O cards such as network controllers. 


. Future Work. 


Most of the future work centers around the FreeBSD kernel (what could be more fun?) and is aimed 
at adding the driver and kernel interfaces to allow detaching and reattaching of drivers/devices, even net- 
work drivers. It is surprising just how many pieces of code this will affect throughout the system. 


On 


Other changes that are planned are: 


- Re-organisation of the slot driver to split it into a slot controller specific portion and device indepen- 
dant portion; this is similar to the structure under DOS/Windows which splits Card Services from 
Socket Services (though the interface will be much narrower). This would also allow different types 
of slot controllers to be managed within the one system. 


= Generation of a separate bus structure to split the PC-Card drivers from the current ISA bus drivers; 
this will allow better integration of PC-Card specific functions such as power management and hot 
swapping, and also provide an interface to the kernel separate to the ISA routines. 

- Incorporation of memory cards, either by a user level NFS process using the read/write interface to 
the card, or by a vnode interface module. 


- Integration of mobile communication support for such things as roaming and IP encapsulation. 


6. Conclusion. 

The PC-Card technology is a fast growing and vital part of the Laptop and mobile computing field. 
Integrating this technology with FreeBSD has been quite easy (well, not as hard as it could have been), but 
further development that impacts considerably more on the FreeBSD kernel will be necessary to obtain the 
most benefit from PC-Cards. This work is progressing, and will be released as part of the FreeBSD project. 

The latest release of this code is available for ftp from dmssyd.syd.dms.csiro.au under 


/pri/mcrae/pemcia-2.0.tgz. 


7. Further Reading. 

The PCMCIA Developers Guide, Michael T. Mori, Sycard Technology, ISBN 0-9640342-0-4 

PC Card Standard - Release 2.01, Personal Computer Memory Card International Association, Sunnyvale 
CA. 
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ABSTRACT 


The Australian Defence Force Activities and Working Hours Survey 1992-1993 collected data 
describing detailed daily activities of over 17,000 uniformed personnel. The information sys- 
tem (IS) supporting this large workplace survey is described and reviewed. A brief introduc- 
tion to the survey background and procedures is presented. Tasks are divided temporally into 
three groups: before, during and after the survey. Before the survey, data from the personnel 
systems of the three forces was loaded into a database and a sample population was selected. 
During the survey, the major IS tasks were to maintain the population data, to track the 
progress of the survey and to facilitate the entry/validation of data being collected. On com- 
pletion of the survey period, the primary IS role was to support the statistical analysis of the 
survey and population data then facilitate document preparation. 


The project is reviewed in terms of the software engineering issues, including the use of a 
‘prototype and evolve’ model for the development of small software modules. The use of 
high-level programming languages and the importance of user feedback in software develop- 
ment are considered. Several coding examples from the survey are included. This case-study 
shows that the UNIX® System, with its range of general-purpose software tools, combined 
with flexible office automation, database and statistical packages provides an effective IS plat- 
form for survey support. 


' formerly with the Directorate of Research, Pay and Conditions Branch, Headquarters — Australian Defence 
Forces, Canberra ACT 2600. 


Disclaimer: views expressed in this article reflect the personal opinions of the authors and not necessarily any official 
policy of the Headquarters of the Department of Defence. 
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1. Introduction 
Methods used to assess the working hours of Australian Defence Force (ADF) uniformed personnel have become 
increasingly rigorous [16]. The ADF Pay and Conditions Branch (PCB) conducted the Activities and Working 
Hours Survey 1992-1993 (AWHS) as a result of a determination by the Chief of Defence Force that a properly 
conducted study of working hours be commissioned. Broadly, the AWHS was 

«initiated as a result of an identified need to develop a management tool for the ADF that provides 

a baseline of data concerning work in the ADF context. The data gathered in this study will enable 

the ADF to present consistent and unbiased data in support of future decisions influencing defence 

structures, pay and working conditions” [1]. 


The AWHS gathered 17,031 responses over a 13 month period. Each response contains in excess of 3531 data 
points. The AWHS collected and analysed over 70Mb of workplace data. 

A number of IS challenges were presented by the AWHS project. The AWHS required the development and 
integration of a tailored software system. Purpose built software modules were produced for tasks including 
sample selection/date allocation, AWHS workflow management, signal drafting, data entry and the preparation 
of data for statistical processing. 

Further, the IS support needed to evolve with the refinement of the AWHS. Since software development was 
never on the critical path for the project, it was practical to prototype and evaluate each software module as its 
need arose. Typically, production software evolved directly from a prototype. Much of the software was devel- 
oped using interpretive languages, particularly Korn-shell (4], perl [13], SQL [2; 7] and Uniplex [3; 11]. 

The AWHS was a success by most measures; running as planned in terms of human resources, time and cost. In 
project management terms, the small AWHS project team was unconventionally structured in that it included 
both users and technical staff. Results from the AWHS were used to support an allowance increase in the 1994 
Service Allowance Case before the Defence Forces Remuneration Tribunal. The value of the survey contribution 
to the case outcome was recognised by extending the AWHS, at a reduced sample rate, on an indefinite basis. 
The success of the project calls for the following description and analysis of the methods employed. Section 2 
describes the computing resources used in the project. The three major phases of the AWHS were the prepara- 
tion, the survey and the data analysis. The IS aspects of these phases are discussed in Sections 3, 4 and 5, 
respectively. Section 6 presents software engineering and IS related results from the survey process. Section 7 
shows some example implementation techniques. Section 8 draws together final remarks and conclusions. 
Analyses of the survey data are presented elsewhere (8; 9; 10; 14; 15]. 


2. Computing Resources. 
The AWHS was adequately equipped with a range of computer hardware and software. The primary hardware 
used by the project is shown in Figure 1. 


Qty Device 

computers: | IBM RS/6000 model 320 

3 IBM PC compatibles 
storage: 2 1Gb disk drives 

2 300Mb disk drives 
tapes: 2 150Mb %" tape 
terminals: 3 X11 terminals 

2 | ASCII terminals 
printers: 1 PostScript laser printer 


Figure 1. Computer hardware. 


The major software system used was IBM AIX version 3.2, a UNIX variant with common System V and BSD 
utilities including ksh(1), sed(1), tr(1), awk/nawk(1), etc. A few advanced users made significant use of the X11 
windowing environment though few X11 applications were used. Non-commercial software used in the project 


included: 
* perl, a string processing language/utility, 2 
*  a2ps, an ASCII to PostScript program, 
*  mtools, programs for handling MS-DOS files systems from UNIX 
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° C-Kermit for file transfers and general remote access. 
Uniplex version 7 was selected as an office automation tool for: 
* word-processing 
* printer control 
* forms and report interface to the database 
¢ its inclusion of a separately accessible SQL (-subset*) database engine. 


An important characteristic of Uniplex software was the use of ASCII files for many internal, and all external, 
file formats. This allows UNIX utilities to transform Uniplex files when required. Uniplex has good UNIX inte- 
gration, allowing the developer to “escape to the shell” at most points where this may be useful. 


SPSS was selected as the statistical package for the AWHS. In the analysis of the survey data, significant use 
was made of both the PC and UNIX versions of SPSS. 


A PC was dedicated to the transmission of signals in the ADF signals network. Signals generated via the survey 
IS were written in batches onto floppy disk then transferred to the signal system. The signals system used soft- 
ware produced by Compucat specifically for checking and transmitting batched signals. 


3. Survey Preparation. 


The commitment to the AWHS from the highest levels of ADF management should be recognised as a major 
contribution to the success of the project. This commitment simplified the organisational politics and reduced 
the negotiation burden inherent at the start of the project. As a result, more initial effort from key project staff 
was available to refine the survey instrument and support mechanisms. 


The survey method [8; 16] required a large number of uniformed personnel to complete a basic form outlining 
major and minor activities of the 48x'4-hour periods of their survey day. Strict criteria for data validity were 
required to meet the statistical goals of the project. Each survey form had to be completed by the right person on 
the specified survey day. Survey data from a respondent would be deemed unusable if it failed to meet the 
administrative requirements of the survey. 


Each survey response was completed by a trained Unit Survey Consultant (USC) who, via a structured interview 
with the respondent, transcribed the respondent’s basic form onto the AWHS form. The survey method and USC 
training were audited by the Australian Bureau of Statistics. The selection and training of 1243 USCs was a 
major task in the survey preparation. 


Once the need for the USCs in the survey process was recognised, the role expanded to include the task of locat- 
ing respondents. The survey IS was to play an important part in supporting the USCs. Coordination of USC 
activity relied on a survey-tracking (or workflow) system which recorded survey events in the database. Work- 
flow was managed through a system of database updates and exception reports based on the (non-)existence and 
temporal relationships of events. Reminder signals were automatically generated and transmitted when neces- 
sary events had not been reported. The characteristics of this system were established prior to USC training. 


To avoid potential data distortion, respondents and their immediate command chain were given as little notice as 
possible of their survey dates. The USCs were given 3 monthly listings of staff and survey dates. Shortly before 
each survey date, the USC was expected to notify PCB that a survey case was ready. The IS was designed to sig- 
nal USCs through military channels when this notification was not received’, 


A major IS task was the tracking of the sample population. Prior to the survey, data was transferred to the survey 
IS from the personnel systems of the three forces. This transfer was designed to be repeated at intervals of 3 
months during the survey. 


We were warned of difficulties encountered in previous attempts to develop a unified ADF personnel database. 
The personnel systems of the three forces are diverse: different data is held, fields vary in format and content and 
fields required by the AWHS are not always present. The AWHS project was able to unify personnel data from 
the three forces because: 

* only part of the data held in the three personnel systems was required; 

*  nochanges to existing personnel systems were required; 


2 Uniplex includes an early Informix database engine which, though limited, was adequate for this project. 
> It appears that some USCs came to rely on this system as a form of personal reminder service. 
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° fields such as RANK were internally recoded without obtaining agreement between the forces as to how 
this should be done; 

* adaptive data transfer methods were used where problems in each transfer were solved as they arose. 
The format of data transfered from the three ADF personnel systems varied both between subsequent data sets 
and between systems, e.g. differences were observed in field separators, field order and date formats. All the 
data was received as ASCII files on 3.5" floppy disks. 
In outline, the process of loading data into the survey IS from each of the personnel systems involved the follow- 
ing steps: 


I ‘eyeball’ the data to determine the field separator then use fr(1) to translate field separators to TABs. 

2. run aperl program over the data to report field sizes and types e.g. text, alphabetic, alphanumeric, numeric 
and various date formats. 

35 use a second perl program to reorganise and reformat fields (especially dates), as well as translate multiple 


spaces to single spaces, remove leading and trailing spaces from fields. 
4. use a third perl program to check the output prior to loading data into the database 
Ds split the data into sections* and load each section into the database. 
6. integrate the new data into the database 
Major database updates were programmed in SQL. Typically, data is loaded into temporary tables. SQL com- 
mands then perform batch updates based on the information in these tables. The temporary tables are then dis- 
carded. An example of this style of processing is given in Section 7. 
Sample selection and survey date allocation used two perl scripts. Skip selects a required number of evenly dis- 
tributed lines from a population file. Randsel makes random selections, without repeats, from the lines of its 
input file(s). An implementation of randsel, with its use in a shell script, is presented in Section 7. Initially, 20% 
of the uniformed personnel as at Ist January 1992 were selected. Where this selected fewer than 80 from an 
employment category, the sample was (randomly) expanded to a maximum of 80 from the category. The size of 
the sample population varied due to discharges and recruitment (Figure 2). 


| Service sample — workforce Jo recruit top-up 
Army 11,522 30,997 37.17 540 
Navy 4,812 15,694 30.66 218 


Air Force Tl 1 21,959 35.11 176 
total 24,044. 68,650 35.02 


Figure 2. Sample and workforce distribution. 


Each member of the survey population is initially allocated a case number. For privacy reasons, the link to the 
case number is broken after the survey is taken. Survey cases were allocated dates and placed in a file ready for 
loading into the database (see Section 7). This resulted in 63 to 65 sample cases per day across the three ser- 
vices. Survey dates were evenly distributed within each service. 

The survey methods were reviewed by representatives of the Department of Industrial Relations and the Aus- 
tralian Bureau of Statistics. Their input improved the quality of the survey instrument and results, and their 
scrutiny was a welcomed contribution to the project. 


4. The Survey. 

The majority of the survey work was performed by the USCs. The largest component of the survey was the 
human resource used to pre-brief and de-brief each survey respondent. USCs were kept informed through a 
newsletter and seminars held around the country. Monitoring of survey progress showed that early detection of 
errors and clarification of USC procedures lead to low levels of problem recurrence. 

USCs were alerted to impending survey cases via two channels. Each USC was issued with a case list every 
three months°®. USCs were required to notify PCB when a case was ready to be surveyed. In 35% of cases, this 
notification had not been received by two working-days prior to the survey date so the ADF signals network was 


+ The old database engine has practical limitations on how much data it can load at one time. 


5 The original survey design issued case lists every two weeks. In trials this lead to high incidents of ‘missed 
cases’ so the release interval was increased for the AWHS. 
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used to convey an official reminder. 


The signals network was used to transmit batches of official communications (signals) to ADF locations, e.g. 
ships at sea. During the course of the survey, over 23,000 signals were transmitted. The highly-structured sig- 
nals were computer-generated either automatically or from stock phrases. Batches of signals were written to 
floppy disk then transmiited by Compucat’s software running on a stand-alone PC connected to the ADF signals 
network. 


USCs were required to notify PCB of personnel relocations affecting staff to be surveyed. Processing the feed- 
back from USCs took 4 PCB person-months for each 3 monthly case-list release. Review of conflicting informa- 
tion arriving from USCs and the ADF personnel systems produced a non-trivial algorithm for resolving informa- 
tion conflicts. Relocation (or posting) information from the USCs in the field was accepted as ‘probably current 
and reliable’ while discharge information from the personnel systems was accurate. 


Data entry and subsequent editing was performed using Korn-shell scripts. The data-entry module was origi- 
nally developed strictly as a prototype. However, the performance and convenience of the module was consid- 
ered comparable to a typical database form-style interface: interactive response was adequate, the user interface 
used fewer keystrokes and the script was perceived as potentially more maintainable than a program written in a 
programming language such as C. Acceptable performance of the data entry module required extensive use of 
Korn-shell built-in string I/O capabilities as outlined in Section 7. 


The inner two pages of the survey form contained the “data matrix”, a unique and distinguishing feature of the 
AWHS [8; 15; 16]. Information in a completed 72x48 data matrix is typically sparse. This sparseness is pre- 
served in the keystroke model and the raw data format used in the project IS. Part of a completed data matrix 
might look like: 


[ 4 B 3 4 


code period 123456789012345678901234567890123456789012345678 


qa land | === 029 9-6----sessHse sss s Sas Ssse assess 
qb Sea 

ge Air 

3a dark 3--------- 323 1--1 

3b outside 


Cn . ——— TT On 


This survey data was entered as: 


row identifier 
data default is 1, unspecified is 0 

3a row identifier 
27-39.3,38.2,41-44.1 data (note: overlapping column ranges) 


and stored in the data-entry file, case . case-number, as: 


_ga 1-48 
_ 3a 27-37 .3,,.38 42739 .3741-44.1 


or alternatively: 


_ga 1-48 
3a 27-39.3,38.2,41-44.1 


This method is efficient in keystrokes and uses relatively little file space (except AIX uses 4k blocks in the file 
system). Each survey form was stored in a separate file, identified by case-number. Directory searching and list- 
ing was slow with all the data files in a single directory. 


The accuracy of the data-entry methods were tested and found to be.high. Average form entry time was 6 min- 
utes. When data entry fell behind, more staff were brought in rather than press the operators for more speed (and 
risk reduced accuracy). 
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5. Data Analysis. 


The final phase of the survey included the analysis of the survey data and the preparation of reports. The large 
volume of survey data meant that it needed to be handled specially. Document preparation and survey analysis 
were expected to be evolutionally: results would influence the direction of the work. 


Personnel data used in demographic analysis is linked to case numbers rather than individual identifiers like “Ser- 
vice Number’. The link between an individual and the survey data was broken, by deleting the case number to 
service number relation, to protect individual privacy. 


The amount of data collected in the AWHS presented problems. Aggregated values and field recoding would 
further increase the size of the dataset. The dataset was too large to be processed practically by SPSS. Another 
method for handling the AWHS data developed. 


The raw data is collected into a single cpio(1) file using using the ASCII header format. This format allows flex- 
ible handling of the data. Individual case files can be extracted for review or exclusion. The perl program, 
xcpio, uses an ‘extraction description file’ (listing data elements, element aggregations and conditional recod- 
ings) and the cpio-format raw data to prepare SPSS input description (‘datalist’) and data files for subsequent 
processing. The extraction process can perform a number of aggregations and conditional recodings which are 
difficult to achieve using SPSS. Since data extraction is relatively efficient, extracted data is treated as temporary 
and an analysis can be repeated based on the data extraction file and the SPSS commands. 


Preparation of large documents can be a problem. The initial survey report [8] included over 100 pages of statis- 
tical analysis and commentary. A wide variety of statistical techniques were used. Other reports [9; 10; 14] are 
of similar size and detail. 


The AWHS reports were prepared using the Uniplex word-processor. Uniplex uses an ASCII file format for its 
word-processing files. A Uniplex word-processing file were constructed from a variety of sources and combined 
into a single ‘document file’. Output from packages including SPSS and SQL were processed using common 
UNIX utilities including sed(1), grep(1), tail(1), awk(1) or perl and embedded in a Uniplex document. Sections 
of Uniplex word-processing text were parameterised using the Korn-shell (like the SQL example in Section 7). 
Parts of the documents were prepared as separate files by different users. Each document section was the output 
of a Korn-shell script and complete documents were constructed hierarchically from smaller components. 


The primary AWHS report [8] was the output from a 15,000 lines of Korn-shell script. The script invokes SPSS 
and SQL commands to include current information from the IS and processed survey results. Document index- 
ing was done using a small perl program prior to formatting by the Uniplex word-processor. 


6. IS Results 


The primary outcomes of this project are the reports produced as part of the project [8; 9; 10; 14] and subsequent 
studies based on the survey data[15]. These results are not repeated here. 


As a survey, the scale of the AWHS is substantial. Each survey response contains 3531 data points (officers 
completed an additional small form). The survey tracked 24,976 and sampled 23,669 uniformed personnel. 
PCB received 18,371 forms achieving a 78% return rate. Of the returned survey forms, 17,031 contained usable 
data (93%). Over the 13 month AWHS, in excess of 60 million data points were collected. With all values pre- 
sent, this would exceed 70Mb of data. As discussed previously, the final data was (is) held in a 13.15Mb cpio(1) 
file (4.13Mb compressed). 

The high return and usable data rates are attributed [8] to the tight integration of the survey into a disciplined 
workplace combined with an effective IS which supported the early identification and correction of problems 
with the survey mechanism and training. Most problems with the survey or its administration were corrected 
quickly by committed and diligent staff in PCB. For the first 6 months, the survey was run by a staff of 7 
(excluding consultants) reducing to 5 for the duration. During the survey, an average of 1.5 data-entry staff were 
used. 


7. Implementation techniques 


In software engineering terms, the AWHS demonstrates the utility of an ‘evolve from prototype’ model as a 
viable software development paradigm for small IS projects. Most of the application software used in the AWHS 
progressed from specification to prototype, then, after user evaluation and minor amendment, into production. 


A principle used in the high-level application design of the survey IS was to keep applications small and simple. 
A simple application can be prototyped using high-level tools (typically a shell script with small perl modules) 
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based on its specification, the data dictionary description of I/O data formats and an initial user interface concept. 
When a prototype meets the actual system requirements of a module, the prototype implementation and its docu- 
mentation replace the low-level design of the module in most (flexibly applied) software engineering paradigms. 
This reduces the development burden and the eventual cost. 


In general, a prototype may not be suitable for production use. The project should learn from this experience. 
When a prototype is unacceptable, a recovery strategy lets the module development process ‘drop through’ to 
conventional design and programming techniques. The prototype review leads to the refinement of the specifica- 
tion. Based on an improved specification, the design and implementation stages have a greater chance of suc- 
cess. 


The recovery strategy was never invoked in the AWHS project; the prototype applications went into the produc- 
tion IS. This is attributed to: 


= the selection of equipment with adequate capacity to meet the demands of the project, 

= a policy of using a small group of flexible software tools rather than specific (commercial) products, 
- direct and effective communication within a small group committed to the success of the project 

= and the open communication with users which is characteristic of a proven implementation group. 


User interface issues were considered critical. The team structure encouraged the attitude that user comments 
and suggestions must be heard and incorporated into project software. In addition to improving overall effi- 
ciency, this contributed to a feeling of involvement in the project. 


The terminal-style user interfaces to the AWHS software modules were, at times, basic and regularly attracted 
comment from visitors to the project. Within the team, the simple interfaces were widely accepted since real 
problems were corrected quickly. As an input device, the keyboard was recognised as more efficient than a 
mouse. Suitable ‘users’ were encouraged to develop basic applications which fostered an appreciation of IS 
issues among the non-technical staff. 


Some of the utilities developed are shown in Figure 3. 


utility language lines description 
[ skip perl 37. select evenly from a file 
randsel perl 21 select randomly from a file 
data perl 131 report field types 
vet perl 132 reformat data files & check fields 
check perl 70 ~~ verify data entry 
form ksh 549 survey specific data entry 
edit ksh 237  edit/correct data entry errors 
demenu ksh 52 front menu for data-entry operators 
xcpio perl 378 data extraction utility 
tgoto Gc 10‘ tput cpu (missing in AIX 3.2) 
other — various small utilities 
total ~3,500 


Figure 3. IS utilities. 


The software maintenance and support effort was minimal. The small code-size of applications meant they were 
easy to develop and maintain. The project could afford an informal approach to software documentation; small, 
simply written modules are easier to understand and require less structural and navigational documentation than 
systems of larger program units. 


Some of the above claims may be difficult to accept unless substantiated. The reader is invited to compare cod- 
ing examples below with other implementation strategies. 


Uniplex SQL can be parameterised and given a command-line interface using a Korn-shell script, e.g. a com- 
mand to delete ‘cases’ or ‘units’ from the database is shown in Figure 4. 


Programs written in perl may look peculiar to anyone unfamiliar with the language. The example in Figure 5 
selects random lines from a file and can be seen as showing how littlé code is needed to create a useful utility, or 
as an example of a useful algorithm expressed in a terse programming language (13, p241]. 
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#!/bin/ksh 
# script to delete database records from list in a file 
# usage : sqldel attribute table delete-file 
te f | ecoss: ] 7 bhen 
echo unable to read file $3 ; exit 1 
fi 
grep "*“$1 " <<-TYPES | read attr tab type 
case casedata char(5) 
unit unitdata char(7) 
TYPES 
Lf { <2 "Stype" =o "$2" lle “Stab" ] 7 then 
echo Sorry, I cannot delete $1 from $2 ; exit 1 
£5. 
# run the parameterised SQL command 
usql <<-SQLCMD 
invoke survey 
create table tmptab ($1 Stype) 
insert into tmptab ($1) paste from $3 
delete from $2 where $1 in (select $1 from tmptab) 
drop table tmptab 
SQLCMD 


Figure 4. Parameterised SQL using Korn-Shell. 


#!/usr/local/bin/perl 


# select random lines from file(s) 
# default - different each time 


srand; 
while(SARGV[0] =~ /*-/) ¢ 
Sesh fee 
/*-sS/ && next; # default 


He 


specify seed 


/*-s(\d+)$/ && (srand($1), next); 
select number (if sufficient) 


/7=n (\ae\ Si && (Sm = Sly mext); 
die “unknown switch: $_\n"; 


te 


} 


# read lines into memory! 


@lines = <>; 

$n = @lines if $n==0 || $n>@lines; # valid $n! 

while(0<Sn--) { # from Perl book 
$line = splice(@lines, rand @lines, 1); 


print $line ; 


Figure 5. Randsel — perl program producing input lines in random order. 


Given case-lists for the three ADF services and a file of 394 dates, dates can be allocated to survey cases using 
the Korn-shell commands in Figure 6. 


for s in army navy raaf 


do 
while : true 
do 
randsel dates 
done | head -$(we -l cases.$s) | pr -s -t -m cases.$s - 


done >casedates 


Figure 6. Korn-shell commands to allocate case dates. 


The perl program, vet, which accepts and reformats data may be one of the most elegant and generally useful 
tools developed in the project. Perl’s ability to treat lists as arrays and to process text files lead to an apparently 
simple implementation. The code in Figure 7 is shows the essence of the reformatting utility, a generalisation of 
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the UNIX cut(1) command. 


#!/usr/local/bin/perl 


# data formatting - extended version of cut(1) 
# options: 
# -ulist list of fields to be converted to uppercase 
# -llist list of fields to be converted to lowercase 
# -dCs input field delimiters are characters Cs 
# -sC output separator is character C 
# -flist output fields in this order 
(Sd Sde, $sep) = ("NE 2 "NE", NEUE # default field delimiters 
Cri= (5 
while(SARGV[0] =~ /*-/) { # process arguments 
So =e SHEE tS 
fs Cal Ll£) Ga)7 && (eval "\@Si"."— = (S2)", next); 
(2d) (oy 8& (Sde = Sil, Sdssl.s2, next); 


je=s(.)/ && (Seen = SL, Next); 
die "unknown switch:$_\n"; 


} 

Sl=i¢ # count from 1 for the users’ sake 

line : while(<>) { # process input (s) 
ery se # multispaces to single space 
s/ *([$da]) */$dc/og; # remove leading & trailing spaces 
Sh Wf 3 


Sfso = splae("S$de", Si)s 
@fields = @_; 
foreach $i (@uf) { 
Strelas(Si}] =" trfa=2/A=Z2/7 
} 
foreach $i (@1f) { 
Sfields (Sal =" tr/A=Z/a=2/; 
} 
# build output list 
@out = @fields; 
LE (S[<=S#EE) - { 
Gout = () 7 
foreach $i (@ff) { 
Sout [++$#out] = $fields[$i]; 
} 
} 
print (join($sep,@out),"\n"); 


Figure 7. Outline of vet — general data reformatting utility written in perl. 


The data-entry module, a Korn-shell script, was one of the more complex modules in the AWHS project. This 
script, developed on a different system, presented some performance and portability problems. The AIX ¢put(1) 
command did not support the cursor positioning capability, cup. A small C program, tgoto.c, (Figure 8), 
was written to overcome this problem. Some other capabilities were named differently on AIX. 


The data entry module could not afford to invoke a separate process (i.e. call tput or tgoto) for each screen 
motion/operation. The data entry script initialised and used string variables as shown in Figure 9. A terminal 
initialisation file is created for each terminal type. When the file is used (in the last line) it establishes string 
variables for cursor positioning. Problems may be encountered where slow terminals require reliable padding in 
the tput(1) output; however modern terminals seem to perform adequately. 
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#include <curses.h> 

#include <term.h> 

main(argc, argv) 
char **argv; 


{ 
setupterm( (char*)0,1, (int*)0); 
putp(tparm(cursor_address,atoi(argv([1]), atoi(argv(2]))); 
exit (0); 
} 
Figure 8. Tgoto.c — AIX replacement for ‘tput cup’. 
if { ! -r S$DIR/term.$TERM ] 
then 
clear="$(clear)" 
echo "$clear\n\nInitial setup.\n\nPlease Wait ...\c" 
clr_eol="$(tput el)" ; clr_eod="$(tput ed)" 
case ‘uname -s’ in 
*AIX*) green="$(tput colf2)" ; white="$(tput GOLEO)" 33 
=) green="$(tput setf 2)" ; white="$(tput Setr 7)" 
Egoto() {( tput cup S* - ) 
esac 
for i in 3456 7 8 8 10 1 12 13 14 15 16 17 18 29 20 
do # spaces-Return padding for slow terminals 
echo lineSi=’"""S(tgote Si 0)"" Naw 
done »>$DIR/term.$TERM 
for i in clear clr_eod clr_eol green white 
do 
echo $i=’"'"Si(eval echo 'S*Sz)"’"* 
done >>SDIR/term.$STERM 
Es 


SDIR/term. $ {TERM} 
Figure 9. Korn-shell performance and portability techniques. 


These string variables are used in Korn-shell commands in the data-entry and editing modules, e.g. 


echo "$linel2${green}Birthday: $white$birthday$clr_eol" 
read qty?"${line20}Quantity: " tmp 


8. Discussion and Conclusions 

Part of the success of the project can be attributed to the dynamics of the group working in the Research Direc- 
torate of the ADF Pay and Conditions Branch. Though the individuals changed with time, the size of the group 
was conducive to effective communication. The structure of the AWHS project group was not the same as 
Brooks’ “Surgical Team” [6], but it exhibited many of the advantages of this organisational model. The inclusion 
of technical staff within the AWHS group tied their objectives to the whole project: a technical group with the 
objective of producing a support IS for the AWHS would normally be larger and less responsive to change 
requests. Within a group, there is a reduced need to negotiate priorities and software developers are responsive 
to user problems. One can conclude that a small project team, including both software developers and users, is a 
desirable organisational structure. 

A purpose-built and evolutionary IS supporting the AWHS has been shown to be effective and economic. Other 
small but complex projects should consider the techniques described here. 

The primary drawback to the ‘prototype and evolve’ style was the difficulty of managing the documentation. 
Documentation evolved with the software and standards were not always adhered to. New project members were 
individually trained and training included indoctrination into the ‘culture’ of the project. 


The project was not constrained by available software. The ability to construct software for specific AWHS 
related problems, e.g. sample-case selection and SPSS data preparation, contributed to the quality of the results 
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aa 


Pills 


produced by the AWHS. The survey design was not compromised by software limitations. 


The use of a UNIX shell as a high-level programming language has been shown to be effective. This is consistent 
with the UNIX model from its inception [12]. The evolution of the UNIX shell [5; 4] and tools like awk/nawk(1) 
and perl [13] make this approach increasingly practical. 
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